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The current school reform effort emphasizes the need to improve the education of all 
students. Assessing the academic achievement of every student is an essential part of 
this reform, but one that presents a challenge for most schools, school districts, and 
states (North Central Regional Educational Laboratory and Council of Chief State 
School Officers, 1996). Traditional assessment practices in many states and school 
districts have tended to exclude students who are learning English as a second 
language. As a consequence, many English language learners (ELLs) are denied 
access to important educational opportunities that are based on assessment results. 

This digest focuses on one type of assessment system, assessment portfolios, and 
discusses the advantages and challenges of using an assessment portfolio system that 
includes ELLs as a district-wide assessment tool. 

WHAT IS AN ASSESMENT PORTEOLIO? 



An assessment portfolio is the systematic collection of student work measured against 
predetermined scoring criteria. These criteria may include scoring guides, rubrics, check 
lists, or rating scales (O'Malley & Valdez Pierce, 1996). Assessment portfolios can 
include performance-based assessments, such as writing samples that illustrate 
different genres, solutions to math problems that show problem-solving ability, lab 
reports that demonstrate an understanding of a scientific approach, or social studies 
research reports that show the ability to use multiple sources. In addition, district-wide 
assessment portfolios can include scores on commercially developed, nationally 
norm-referenced tests, such as the Iowa Test of Basic Skills, or results of state 
assessment measures, such as the Maryland School Performance Assessment 
Program, as well as other information pertaining to students' educational backgrounds. 

WHAT ARE THE ADVANTAGES AND 
DISADVANTAGES OF ASSESSMENT 
PORTFOLIO 

SYSTEMS FOR ENGLISH LANGUAGE LEARNERS?lnclusion of English language 
learners 

Unlike other assessment programs, assessment portfolios do not exclude certain 
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Student populations. Many states have policies that restrict English language learners 
from taking commercially developed, norm-referenced and criterion-referenced tests for 
up to 3 years after their arrival in a U.S. school system. In contrast, all students can be 
included in an assessment portfolio system. 



Increased school accountability for all students 

At the state level, assessment information is often collected to ensure that the 
educational system addresses the needs of all students. Inclusion of ELLs in state and 
local testing programs is critical to accountability and to providing accurate data about 
the achievement of these students (O'Malley & Valdez-Pierce, 1996). By providing 
systems with a richer source of information about school learning, assessment portfolios 
can help school systems identify and meet the needs of diverse students, including 
ELLs. 



A shared vision of student goals and learning 

By developing an assessment portfolio system that includes English language learners, 
teachers, administrators, parents, and students can shape a common vision of what 
students should know and be able to do as a result of their course work. By clearly 
articulating expectations and the criteria upon which to assess attainment of these 
expectations, school systems help create a shared vision of the purpose of education 
based on the values of the community. 



Authentic picture of learning 

Assessment portfolios can be designed to measure virtually any observable skill or 
process or content-area knowledge needed for system-wide assessment purposes. A 
wide range of student products can be included in assessment portfolios as long as 
predetermined scoring criteria are in place. Portfolios are designed to be inclusive of all 
students and to provide an authentic description of what students can do. 



Improved teaching and student learning 

Using assessment portfolios that include English language learners not only provides 
improved information about student achievement but also makes a positive impact on 
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teaching and student learning. According to Geoff Hewitt, a writing assessment 
consultant, when teachers are trained to use and score portfolios based on 
agreed-upon criteria, they tend to move toward a more learner-centered teaching 
model, which encourages students to take more responsibility for their own learning 
(personal communication, October 1 1 , 1996). Through such training, teachers develop 
an understanding of the quality of student work that meets specific achievement levels 
according to the scoring criteria. 



Reflection of assessment reform 

Advocates of assessment reform call for new measures that provide a better 
understanding of student achievement, especially for English language learners. By 
using assessment portfolios that include ELLs, school systems could reduce the 
number of students excluded from system-wide assessment and possibly increase the 
number of teachers participating in professional development activities. 

WHAT ARE THE CHALLENGES OE 
ASSESSMENT PORTFOLIOS THAT INCLUDE 
ENGLISH 



LANGUAGE LEARNERS?Lower comparability and reliability 

Many performance-based tests, including some portfolio systems, do not easily or 
meaningfully translate into a single score or set of scores. The public has become 
accustomed to single scores, such as those used to describe the results of standardized 
or norm-referenced tests. Single scores are comparable across systems and from one 
year to the next. Because some districts report the outcomes of performance-based 
tests in words rather than numbers, some stakeholders feel the school system is less 
accountable for individual students. 

In addition, it is difficult to implement assessment portfolios that meet the reliability 
requirements many school systems want. Achieving a certain degree of reliability 
among raters or test evaluators is important (Novack, Herman, & Gearhart, 1996). 
Without high inter-rater agreement figures, the usefulness of the scores as an 
accountability tool diminishes, because the results cannot be used to compare scores 
reliably between schools or over time. 



Difficulty ensuring standardized testing conditions 
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Some states and districts are prevented by law from implementing performance-based 
assessments that include portfolios. Some state legislatures mandate the use of 
traditional norm-referenced tests because of the perception that standardized tests 
require students to perform under similar circumstances (Special Issues Analysis 
Center, 1995). When using portfolio assessments, performance conditions may vary, 
and teacher bias can affect students' performance. For example, the amount of support 
teachers provide to students, the amount of time students are allowed to spend on 
portfolio samples, and the extent to which student work is augmented by support from 
external sources have raised questions about the validity of inferences about student 
competence based on portfolio work (Gearhart, Herman, Baker, & Whittaker, 1993). 



Cost 

One hurdle to developing an assessment portfolio system is the ongoing expense of the 
program. Designing, implementing, and scoring portfolio items is labor intensive and 
therefore costly. Practitioners invest a substantial amount of time to align the 
assessment tasks with the curriculum and to develop the scoring criteria and scoring 
tools. Persuading state legislatures and school boards to agree to the additional costs 
involved in scoring an assessment portfolio system that includes ELLs can be difficult. 



Scoring 

Portfolios pose a difficult scoring dilemma for three reasons. First, developing and using 
scoring criteria requires labor intensive staff discussion and training. When ELLs are 
included in a portfolio system, the scoring must be designed to assess content 
knowledge and language proficiency. Second, inter-rater agreement among teachers 
and other scorers requires intensive staff development. Third, scoring student work 
using predetermined criteria is more time consuming than scoring a single-occasion, 
norm-referenced test. Fortunately, these same activities also lead to improved teaching 
and learning. 

STEPS TO DEVELOPING AND IMPLEMENTING 
AN ASSESSMENT PORTEOLIO SYSTEM 

THAT INCLUDES ENGLISH LANGUAGE LEARNERS:"Decide about goals and 
content." Stakeholders in the school and school district, including those responsible for 
English language learners, decide what assessment information is needed and how that 
information can be provided. A group of teachers agree to lead the program. 
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"Design the portfolio assessment program." The lead group of teachers, including 
English as a second language and bilingual education teachers, administrators, and 
parents decide on the range of products to be included in the portfolio assessment 
program. 

"Develop scoring criteria and standards of performance." The group decides on 
common goals for student learning and performance and how they will be assessed, 
develops scoring rubrics and checklists, and agrees on standards of performance to be 
attained. If possible, benchmarks that exemplify student work are articulated, including 
benchmarks for English language learners. 

"Align tasks to standards and curriculum." The group aligns the assessment tasks to the 
district or state content standards and curriculum frameworks. 

"Implement at pilot sites, provide staff development, and analyze results." Decide on 
pilot sites and provide staff development on the implementation of portfolios. Following 
the implementation at pilot sites for at least one full school year, score the portfolios 
from the pilot sites. Assess the effectiveness of the program and modify the scoring 
criteria based on feedback from the pilot site educators and results of the scored 
portfolios. Study the effects of the program on English language learners in particular to 
determine whether improved information is available as a result of the portfolio 
implementation. 

"Implement at all sites." Once the program has been piloted and found to be effective, 
implement the portfolio program at all sites. 

"Train teachers to score." Train a team of teachers to score student work using the 
portfolio program's scoring criteria and benchmarks. Training should include discussion 
of second language proficiency and its impact on student achievement. Efforts are 
made to reach an inter-rate reliability level of .7-. 8. 

"Establish guidelines for administration." Stakeholders develop guidelines for a 
standardized collection of student work and decide on the time, place, and manner in 
which standardized prompts will be given to assess students throughout the system. 
Accommodations for English language learners are delineated. 

"Score the portfolios." Teachers score the portfolios based upon predetermined criteria. 
This is typically done over several days in a central location by teachers who have been 
trained. 

"Report the results." All stakeholders receive information about the results of the 
portfolio assessment in a timely fashion in ways that make the results meaningful to 
everyone, including teachers, students, parents, and other community members. 

"Evaluate the program." After one year, evaluate the effectiveness of the portfolio 
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program and make necessary judgments. 

CONCLUSION 



Developing and implementing a large-scale assessment portfolio program that includes 
English language learners requires extensive planning and discussion and considerable 
resources. It also offers considerable advantages. Stakeholders within the system have 
a common vision about what students should learn and be able to do, how goals will be 
assessed, and what criteria will be used. Improved teaching and learning are natural 
outcomes of a well-designed, well-implemented assessment portfolio system. 
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This Digest is based on an article published by the Northeast and Islands Regional 
Educational Laboratory at Brown University in Perspectives on Policy and Practice 
(March 1999): "Creating Large-Scale Assessment Portfolios That Include English 
Language Learners" by Emily L. Gomez. The research for the article was completed in 
1997. 
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